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TITLE OF THE INVENTION 

IMAGE REPRODUCING DEVICE AND IMAGE 

RECORDING/REPRODUCING DEVICE 

5 BACKGROUND OF THE INVENTION 
(1) Field of the Invention 

The present invention relates to an image recording device 
such as a digital still camera and a video camera as well as to an 
image reproducing device for playing still images and moving 
10 images (to be referred to comprehensively as an "image" 
hereinafter) shot by such image recording device, and to a 
technique to facilitate a selection of an image when reproducing an 
image. 

15 (2) Description of the Related Art 

In recent years, there has been a notable diffusion of an 
image recording device such as a digital still camera and a digital 
video camera. With the spread of these image recording devices, 
various kinds of suggestions are presented for a usability-oriented 

20 method to manage image data using an image reproducing device 
for reproducing data of a shot image. For example, an image 
management system for displaying camera parameters which are 
set at the time of shooting, the date and time of shooting, weather 
information at the time of shooting and other information is 

25 disclosed in Japanese Laid-Open Patent Application No. 
2002-10178. 

Similarly for the image recording device, diverse suggestions 
are also made for a technique to facilitate the image data 
management after shooting. For example, a camera device to 
30 detect its own position (camera position) using a positioning unit, 
calculate an object position using a camera-object distance 
retention unit and a direction in which the object is shot and record 
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the camera position, the object position and the image data by 
associating these positions with the image data is disclosed in 
Japanese Laid-Open Application No. 2001-169164. 

However, the image management system mentioned above 
5 contains a problem that a user cannot search for a desired image 
visually and intuitively on the screen to select an image because 
labels indicate only time and place where an image is shot are 
attached on the screen to select an image. Another problem is that 
the user can neither search out quickly an image that is frequently 

10 shot or an image that is frequently reproduced nor find out easily an 
image that is rarely shot or an image that is rarely reproduced. 

On the other hand, the camera device mentioned above 
contains a problem that the user cannot calculate a right position of 
the object which the user desires to shoot because of the errors 

15 made by the camera parameters. There is a risk that the user 
might shoot an object different from the one desired by the user. 

SUMMARY OF THE INVENTION 

The first object of the present invention, conceived in view of 
20 above problems, is to provide an image reproducing device for 
facilitating a selection of an image. 

The second object is to provide an image recording device 
capable of calculating a right position of an object desired by the 
user and of obtaining a position of the object desired by the user 
25 even when plural objects are shown in the range of shooting in 
shooting an image. 

In order to achieve the above objects, the following technical 
measures are taken. 

Namely, the image reproducing device according to the 
30 present invention for reproducing image data and outputting the 
reproduced image data to a display device, the image reproducing 
device comprises: an input unit operable to obtain the image data, 
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and image information including position information relating to at 
least one of a shooting position from which the image is shot and an 
object position included in the image data; a map display unit 
operable to display a map on the display device; and a position 
5 display unit operable to display a symbol indicating the image data 
in a position corresponding to either the shooting position or the 
object position on the map, based on the position information 
included in the image information. Thus, the position in which the 
image is shot and the position of the object are visually displayed on 

10 the map using symbols such as icons or the like, therefore, it 
becomes easier to select a desired image. 

The image reproducing device further comprises: an image 
information analysis unit operable to analyze the obtained image 
information; and a symbol attribute change unit operable to change 

15 at least one attribute of the symbol based on the analysis result 
obtained by the image information analysis unit, wherein the 
position display unit displays the symbol according to the attributes 
changed by the symbol attribute change unit. Thus, the size and 
the color of the symbols like the icons to be displayed by the display 

20 device are changed according to the shooting frequency. Therefore, 
the user can visually and intuitively search for the desired image 
that is frequently shot. The image information may include 
information on a length of time for shooting the image data so that 
the image information analysis unit analyzes the length of shooting 

25 time for each image data and the symbol attribute change unit 
changes the symbol based on the length of shooting time. 

The image reproducing device further comprises: a 
reproduction information measurement unit operable to measure at 
least one of the following: the number of times reproducing the 

30 image data and a length of time taken for reproducing the image 
data; and a reproduction information retention unit operable to add 
up each measurement result obtained by the reproduction 
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information measurement unit and retain the obtained addition 
result, wherein the symbol attribute change unit changes at least 
one attribute of the symbol based on the addition result retained by 
the reproduction information retention unit. Thus, the size and the 
5 color of the symbols like the icons to be displayed by the display 
device are changed according to the reproduction (playback) 
frequency and the length of reproduction (playback) time. The 
user can therefore search out quickly the image that is frequently 
reproduced. 

10 An image recording/reproducing device according to the 

present invention, which includes a camera for shooting an object 
and generating image data, for reproducing the generated image 
data and outputting the reproduced image data to a display device 
comprises: a camera position/direction detection unit operable to 

15 detect a camera position and a direction of the camera; an object 
distance detection unit operable to detect a distance between the 
camera and the object based on a focal length of the camera; an 
object position calculation unit operable to calculate a position of the 
object based on the detected camera position and camera direction, 

20 and the detected distance to the object; an object search unit 
operable to search for a plurality of objects within a predetermined 
spatial range having the calculated object position in the center; a 
candidate object selection unit operable to select a single object 
from among the plurality of objects, when a plurality of objects are 

25 found as a result of the search conducted by the object search unit; 
a shooting unit operable to shoot the selected object and generate 
image data; a storage unit operable to store the generated image 
data and image information including position information relating 
to at least one of the camera position and the object position; a map 

30 display unit operable to display a map on the display device; and a 
position display unit operable to display a symbol indicating the 
image data on a position corresponding to either the camera 
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position or the object position on the map. Thus, a right position of 
the desired object can be obtained even when plural objects are 
presented as candidates in shooting an image. 

The present invention can be realized not only as such image 
5 reproducing device or image recording/reproducing device but also 
as an image reproduction method or an image 
recording/reproduction method having the characteristic units 
included in these devices as steps, and even as a program causing a 
computer to execute these steps. Needless to say, such program 

10 can be distributed via a storage medium such as a CD-ROM or a 
transmission medium like Internet or the like. 

As is described above, the image reproducing device 
according to the present invention counts a shooting frequency of 
the image shot in the same location and a length of shooting time of 

15 the image data as well as a reproduction frequency and a length of 
reproduction time of the image data, retains the records and 
changes the size, color, and form of the icons according to the 
shooting frequency, the length of shooting time or the reproduction 
frequency and the length of reproduction time in displaying the icons 

20 for indicating the image data on the map. The preference on the 
image, therefore, can be displayed visually and thereby the process 
of selecting an image can be facilitated. 

Furthermore, the image recording/reproducing device 
according to the present invention detects a camera position, a 

25 camera direction and camera parameters, calculates an object 
position based on them and then, searches whether or not plural 
candidate objects are found within the predetermined spatial range 
having the calculated object position in the center. The user can 
surely select a desired object since a single object can be selected 

30 when plural objects are found as candidates to be shot. The camera 
parameters optimal for shooting can be set or the camera position 
can be always corrected, using the position and the size of the 
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selected object, so that a framing at the time of shooting can be 
facilitated. 

The image recording/reproducing device according to the 
present invention can also display visually the shot image 
5 corresponding to the position where the image is shot or the position 
of the object on the map, as in the case of the image reproducing 
device. In addition, the preference on the image can be displayed 
visually because the icon on the map indicating the image data can 
be changed according to the shooting frequency and the length of 

10 shooting time. What is more, the image recording/reproducing 
device displays planned shooting locations on the map so that the 
user is informed of the sites for shooting indicated on the map as 
well as the icons indicating the sites where shooting is not 
performed yet so that the user will not miss the shootings to be 

15 carried out. 

Concluding from what is described above, the image 
reproducing device according to the present invention displays the 
information on the shot image together with a map and further 
displays the icons emphasized according to the shooting frequency 

20 and the length of shooting time or the reproduction frequency and 
the length of reproduction time. The user can therefore easily 
manage the image data and its practical value is very high. 

For further information about technical background to this 
application, Japanese application No. 2003-009460 filed 17 January, 

25 2003, is incorporated herein by reference. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other subjects, advantages and features of the 
invention will become apparent from the following description 
30 thereof taken in conjunction with the accompanying drawings that 
illustrate a specific embodiment of the invention. In the Drawings: 

FIG. 1 is a diagram showing an outer appearance of an image 
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reproducing device according to a first embodiment; 

FIG. 2 is a block diagram showing a hardware configuration of 
the image reproducing device according to the first embodiment; 

FIG. 3 is a block diagram showing a functional structure of the 
5 image reproducing device according to the first embodiment; 

FIG. 4 is a tree diagram showing an example of an option 
relating to settings of a display method using icons and symbols; 

FIG. 5 is a tree diagram showing an example of an option 
relating to level settings; 
io FIG. 6 is a tree diagram showing an example of an option 

relating to icon change attribute settings; 

FIG. 7 is a flowchart showing a flow in displaying an image 
selection screen based on shooting information; 

FIG. 8 is a flowchart showing a flow in displaying an image 
15 selection screen based on information relating to a reproduction of 
an image; 

FIG. 9 is a diagram showing an example of data structure for 
information relating to a shot image; 

FIG. 10 is a diagram showing an example of data structure for 
20 information relating to a reproduced image; 

FIG. 11 is a diagram showing an example of display on the 
image selection screen based on the information relating to the shot 
image; 

FIG. 12 is a diagram showing an example of display on the 
25 image selection screen based on the information relating to the 
reproduced image; 

FIG. 13 is a diagram showing an example of the image 
selection screen presenting images which are shot in the same 
position; 

30 FIG. 14 is a diagram showing an example of a display method 

when a shooting site is shifted sequentially for many times; 

FIG. 15 is a block diagram showing a hardware configuration 
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of the image recording/reproducing device according to a second 
embodiment; 

FIG. 16 is a block diagram showing a functional structure of 
the image recording/reproducing device according to the second 
5 embodiment; 

FIG. 17 is a diagram showing processing of calculating an 
object position; 

FIG. 18 is a diagram showing a result of detecting plural 
candidate objects; 
10 FIG. 19 is a diagram for describing processing of correcting a 

camera position; 

FIG. 20 is a pattern diagram showing a camera position, an 
object position, and a size of the object; 

FIG. 21 is an example of a display of an object that is shot 
15 after correcting its camera direction; 

FIG. 22 is a flowchart showing a flow of correction processing 
performed by the image recording/reproducing apparatus according 
to the second embodiment; 

FIG. 23 is a diagram showing an example of a screen for 
20 selecting a candidate object; 

FIG. 24 is a diagram showing an example of a screen after 
selecting an object; 

FIG. 25 is a diagram showing an example for a display of 
planned shooting locations. 

25 

DESCRIPTION OF THE PREFERRED EMBODIMENT(S) 

The following describes in detail the embodiments of the 
present invention with reference to the drawings. 

30 (First Embodiment) 

Firstly, an image reproducing device according to the first 
embodiment of the present invention will be described. 
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FIG. 1 shows an image reproducing device according to a first 
embodiment. 

The image reproducing device 100 can be a home-use AV 
server apparatus or a computer apparatus for obtaining image data 
5 from an Internet 10 or the like, reproducing the obtained image data, 
and outputting the reproduced image data for display to a display 
device 110 such as a TV for display. 

The image reproducing device 100, connected to the Internet 
10, downloads the image data transmitted from the Internet 10 or 

10 reads the image data shot by a digital still camera 21 or a digital 
video camera 22 recorded on a memory card, or a storage medium 
30 such as a memory card, a CD-ROM, a DVD-ROM, and the like, or 
reads the image data directly from the digital still camera 21 or the 
digital video camera 22 via a cable or wireless connected to the 

15 image reproducing device 100, and reproduces it according to the 
instructions from the user which are received via a remote controller 
120. 

FIG. 2 is a block diagram showing the hardware construction 
of the image reproducing device 100 according to the first 

20 embodiment. 

As shown in FIG. 2, the image reproducing device 100 
includes a CPU101, a memory 102, an image/shooting information 
input interface 103, a drawing input interface 104, an I/O 105 and a 
drawing display/output interface 106. 

25 The image/shooting information input interface 103 is an 

interface such as a Universal Serial Bus (USB) port and an IEEE1394 
port, and the like, for inputting the image data and the shooting 
information included in the image data. Obtaining the image data 
via Internet 10 can be realized with the use of a LAN port or the like. 

30 The image/shooting information input interface 103, whether linked 
by wired or wireless connection, downloads the image data as well 
as shooting information into the device by communication with a 
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camera or the like using electrical or digital signals. 

The drawing input interface 104 is a drive device or the like 
for reading a drawing such as map data stored in a storage medium 
such as a CD-ROM and a DVD-ROM and inputting it in the image 
5 reproducing device 100. The drawing input interface 104 may 
obtain the map data when ever necessary required from other 
devices, via the Internet 10 or the like not only from the storage 
medium as described above. This can be realized with a modem or 
a LAN port, or the like. The map data presented as a drawing is 
10 image data indicating topography and vector data representing 
roads, lines, buildings and others. 

The I/O 105 can be realized with the remote controller 120 for 
receiving operations from the user and the buttons placed on the 
device. 

15 The drawing display/output interface 106 can be realized with 

a graphics chip, or the like, equipped with a Video Random Access 
Memory (VRAM) for creating a screen to be outputted to the display 
device 110 based on the instructions from the CPU 101. 

FIG. 3 is a block diagram showing a functional structure of the 

20 image reproducing device 100 according to the first embodiment. 

As shown in FIG. 3, the image reproducing device 100 
includes an image input unit 131, an image/shooting information 
storage unit 132, a image/shooting information analysis unit 133, a 
display method selection unit 134, a drawing/map display unit 135, 

25 a drawing/map reading unit 136, a position display unit 137, a 
symbol attribute change and display unit 138, a selected image 
reproduction unit 139 and a reproduction information measurement 
and retention unit 140, as functional components. 

The image input unit 131 is a processing unit for reading in an 

30 image shot by a digital still camera or the like, camera parameters 
such as a picture angle, a focus and an aperture value, which are set 
for a camera at the time of shooting, and the shooting information 
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on a shooting position and an object position or the like, and can be 
realized with the image/shooting information input interface 103 
shown in FIG. 2. 

The image/shooting information storage unit 132 can be 

5 realized with a storage device such as a video tape, a memory, a 
hard disk or the like for storing the image inputted by the image 
input unit 131, the camera parameters and the shooting information 
such as shooting position and object position. 

The image/shooting information analysis unit 133 is a 

io processing unit to count the number of images which are shot from 
the same shooting position or which have the same object positions 
and to sum up the length of shooting time, by analyzing the shooting 
information stored in the image/shooting information storage unit 
132. Whether or not the positions are same is determined based on 

15 a range specified for the judgment of the position by the user via the 
I/O 105. The positions are determined as same when they are 
found within the specified range. The shooting position and the 
object position may be received as an input from the user via the I/O 
105, and the range described above may be a value that is 

20 programmed beforehand. 

The display method selection unit 134 is a processing unit to 
receive a selection of a display method concerning how to display 
the information indicating the image data stored in the 
image/shooting information storage unit 132 on the screen for 

25 receiving a selection of an image to be reproduced from the user 
(image selection screen). Here, the information indicating the 
image data can be, for instance, icons and symbols, and the display 
method to be selected can be one of the following : whether or not to 
display the shooting position of the image and the object position 

30 included in the image data as icons on a map; whether or not to 
display a length of shooting time or the like by changing the size of 
the icons; and whether or not to display a moving trace or the like by 
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modifying the map to be displayed by the drawing/map display unit 
135. The display method selection unit 134 receives a selection of 
the display method on the image selection screen out of several 
types from the user via the I/O 105. When receiving the selection, 
5 the display method and shifting method can be assigned 
respectively to the numbers and arrow signs indicated on the remote 
controller 120, for instance, so that the inputs are received via the 
remote controller 120. 

FIG. 4 is a diagram showing an example of a tree diagram 
10 showing an option relating to the display method settings for icons 
and symbols. 

For the display of a "map", an option such as "standard" and 
"with degrees" (whether to change the size and the color) is 
available. When the display method is "standard", the map is 
is displayed in a homogeneous size whereas when the display method 
is "with degrees" the map is displayed in partially changed size and 
color. 

For the display of a "moving trace", an option either "display" 
or "non-display" is available. When "display" is selected, options 
20 such as "standard" and "with degrees" are available. When 
"standard" is selected for the display method, the moving trace is 
displayed in a homogenous size, however, when the display method 
is "with degrees" the moving trace is displayed in partially modified 
size and color. 

25 For the "shooting position" and "object position", the same 

options are available as shown for the "moving trace". 

FIG. 5 is a diagram showing an example of a tree diagram for 
an option relating to degree standard settings when "with degrees" 
is selected in FIG. 4. 

30 The "degree standard settings" is a standard for changing the 

information indicating the image data on the image selection screen, 
and can be selected from among "time", "frequency" and "time + 
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frequency". When "time" is selected, a length of shooting time and 
a length of reproduction (playback) time of the image data are 
determined as a degree standard and when "frequency" is selected, ~> 
an amount of image data shot from the same position as a camera 
5 position or shot in the same object position, namely, shooting 
frequency and reproduction (playback) frequency of the image data 
are determined as a degree standard. When "time + frequency" is 
selected, a combination of shooting time or a length of reproduction 
time and shooting frequency or reproduction frequency is 

10 determined as a degree standard so that the image selection screen 
is displayed by changing the display of the icons. When time and 
frequency are combined, the number of frequency to which the 
value obtained by dividing a time by a certain unit of time is added 
is determined as a degree standard. 

15 FIG. 6 is a diagram showing an example of a tree diagram for 

an option relating to icon change attribute settings when "with 
degrees" is selected in FIG. 4. 

Either "size", "brightness" or the both can be selected in 
setting the "icon change attributes" and either "proportion," 

20 "inverse proportion" or "constancy" can be selected for the degree 
standards. When "proportion" is selected, the size and the 
brightness of the icon are changed in proportion to the degree 
standard and when "inverse proportion" is selected, they are 
changed in inverse proportion to the degree standard. And when 

25 "constancy" is selected, the icons are displayed on the image 
selection screen with the size and the brightness set as constant, 
irrespective of the degree standard. Color or form may be inserted 
as an item under the icon change attributes. 

Thus, a degree display can be determined based on the 

30 "degree standard" and the "icon change attributes". The degree 
display may be set respectively for each of the elements shown in 
FIG. 4: map, moving trace, shooting position, and object position. 
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It may be set for all the elements together. The types of placing 
and displaying options shown in FIGS. 4~6 are only the examples of 
a display method, and thereby shall not be restricted to them and 
may be presented in a different way. 
5 The drawing/map reading unit 136 is a processing unit for 

reading drawings like map data stored in a storage medium such as 
a CD-ROM and a DVD-ROM and can be realized with a drawing input 
interface 104 shown in FIG. 2. 

The drawing/map display unit 135 is a processing unit for 

10 displaying the drawing read by the drawing/map reading unit 136 on 
the screen of the display device 110 and can be realized with the 
drawing display/output interface 106 shown in FIG. 2. When the 
display method selection unit 134 sets to display a shooting position 
or the like of the image data, the drawing/map display unit 135 

15 display, on the display device 110, the map corresponding to the 
position that is already set. When the display method selection unit 

134 sets to display a moving trace, the drawing/map display unit 

135 display, on the display device 110, with the use of lines the 
moving trace obtained by capturing the camera position or linking 

20 the shooting positions in sequence. 

The position display unit 137 is a processing unit for 
displaying a shooting position of the image and an icon 
corresponded to the image data or a position of the object included 
in the image, based on the information on the shooting position of 

25 the image analyzed by the image/shooting information analysis unit 
133 or the position of the object included in the image. The 
position display unit 137 is also a unit for displaying the icon 
corresponded to the image by placing it on the map to be displayed 
on the screen, and can be realized with the drawing display/output 

30 interface 106 shown in FIG. 2. 

The selected image reproduction unit 139 is a processing unit 
for making the icons indicating a shooting position and an object 
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position displayed on the screen of the display device 110 selectable 
with the use of the I/O 105 such as the remote controller 120, 
reading the image data corresponding to the selected icon from the 
image/shooting information storage unit 132 and outputting it to the 
5 display device 110, and can be realized with the drawing 
display/output interface 106 or the like. 

The reproduction information measurement and retention 
unit 140 is a processing unit for measuring the reproduction 
frequency of the image selected by the selected image reproduction 

10 unit 139 and outputting it to the symbol attribute change and 
display unit 138. 

The symbol attribute change and display unit 138 is a 
processing unit for obtaining the shooting information such as the 
shooting frequency and the length of shooting time outputted by the 

15 image/shooting information analysis unit 133 and as the shooting 
position and the object position as well as reproduction information 
such as the length of reproduction time and reproduction frequency 
outputted by the reproduction information measurement and 
retention unit 140, and for changing the drawing and the moving 

20 trace to be displayed by the drawing/map display unit 135, the color, 
the form, the size or the like of the icons to be displayed by the 
position display unit 137, according to the display method set by the 
display method selection unit 134. The symbol attribute change 
and display unit 138 changes the color, form and size of the lines and 

25 the icons in the drawing depending on the shooting frequency and 
the length of shooting time in accordance with "size" and 
"brightness" of the icon change attributes. In this case, when 
"proportion" is selected as a degree standard, the changes are made 
in proportion to shooting frequency and shooting time, for example, 

30 the line is thickened, the icon is enlarged or the color is brightened. 
In contrast, when "inverse proportion" is selected, the changes are 
made in inverse proportion to shooting frequency, a length of 
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shooting time, or the like, for example, the line is thinned down, the 
icon is made smaller, the color is toned down. When "constancy" is 
selected, the icons are displayed with the initial values set for the 
attributes. Here, the symbol attribute change and display unit 138 
5 may change the display of the icons in proportion to or in inverse 
proportion to logarithms obtained for the frequency and the length 
of time instead of directly to the frequency and the length of time to 
which constant upper and lower limits can be set so that the value is 
controlled. 

10 Furthermore, the colors of icons and the lines may be 

brightened keeping the same tone, the colors may be changed 
keeping the same brightness or the color of patterns may be 
changed. 

Next, an operation performed by the image reproducing 
15 device 100 constructed as above is described with reference to FIGS. 
7 and 8. 

FIG. 7 is a flowchart showing a flow of processing when the 
image reproducing device 100 displays an image selection screen 
based on the shooting information. 

20 The image input unit 131 firstly reads the image data shot by 

a digital still camera or the like and the shooting information such as 
camera parameters, a shooting position and an object position 
included in the image data (S100) and stores them in the 
image/shooting information storage unit 132 (S102). 

25 The image/shooting information analysis unit 133 then 

analyzes the shooting information stored in the image/shooting 
information storage unit 132 and extracts the shooting position and 
the object position of the image data (S104). Here, the 
image/shooting information analysis unit 133 counts the number of 

30 images whose shooting positions or the object positions are the 
same and outputs the result to the symbol attribute change and 
display unit 138. 
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The display method selection unit 134 also receives selections 
from the user on how to display the icons indicating either the 
shooting position or the object position, i.e.) size, color, form, and 
also the drawing, and assigns a display method according to the 
5 selections (S106). 

Subsequently, the drawing/map reading unit 136 reads the 
drawing such as map data (S108) whereas the drawing/map display 
unit 135 and the position display unit 137 display out, on the display 
device 110, the image selection screen with the icons which have 

10 been changed according to the assigned display method (S110) and 
terminates the processing. 

FIG. 8 is a flowchart showing a flow of processing when the 
image reproducing device 100 displays an image selection screen 
based on the reproduction information. 

15 When the display device 110 displays the image selection 

screen with icons (S120), the selected image reproduction unit 139 
firstly receives an operation of selecting an icon (Yes in S122), reads 
out the image data corresponding to the selected icon from the 
image/shooting information storage unit 132 for reproduction and 

20 outputs it to the display device 110 (S124). 

The reproduction information measurement and retention 
unit 140 measures the length of time taken for reproducing an 
image, and then, updates the reproduction frequency and the length 
of reproduction time when the reproduction of the selected image 

25 data is terminated (S126) and outputs the updated records to the 
symbol attribute change and display unit 138. 

Subsequently, the symbol attribute change and display unit 
138 updates the display of the icons based on the updated 
reproduction frequency and the length of reproduction time (S128). 

30 The image reproducing device 100 repeats the above 

processing until it obtains an instruction to exit the screen for 
selecting an image (No in S130) and terminates the processing when 
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it obtains the instruction (Yes in S130). 

The following describes in detail an image selection screen to 
be displayed by the display device 110. 

Before explaining the image selection screen, the information 
5 used by the image reproducing device 100 to create an image 
selection screen will be clarified. 

FIG. 9 is a diagram showing an example of data structure of 
shot image information. 

Shot image information 151 is generated by the fact that the 
10 image/shooting information analysis unit 133 analyzes the shooting 
information. The shooting position, the object position and the 
length of shooting time are written for each image number. 

FIG. 9 shows shot image information 151 relating to four 
types of images from an image number Al to an image number A4. 
15 As shown in the diagram, the object positions of the image A2 and 
the image A4 are on the same axis, therefore, they are the images 
in which the same object is shot. The length of the time taken for 
shooting the image Al and the length of the time taken for shooting 
the image A3 are the same and thereby indicate the same value. 
20 FIG. 10 is a diagram showing an example of data structure of 

reproduced image information 152. 

The reproduction information measurement and retention 
unit 140 creates and updates the reproduced image information 152 
based on the result obtained from the reproduction performed by 
25 the selected image reproduction unit 139. The length of 
reproduction time and the reproduction frequency are written for 
each image number. 

FIG. 10 shows the reproduced image information 152 relating 
to four types of images from an image number Al to an image 
30 number A4. As shown in the diagram, the image Al is reproduced 
the most frequently and for the longest time. 

FIG. 11 shows a screen to be outputted for display by the 
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display device 110 based on the shot image information shown in 
FIG. 9. 

The first screen example 160 shown in FIG. 11 is an example 
of a display on the image selection screen to which the image 
5 reproducing device 100 has added changes in order to display the 
icons larger according to the length of shooting time. 

In the first screen example 160 outputted based on the shot 
image information, an image with the image number Al is 
represented by an icon of video camera 161 whereas images with 
10 the image numbers A2 and A4 are represented by an icon 163 and an 
image with the image number A3 is represented by an icon 162. As 
shown in FIG. 9, the images with the image numbers A2 and A4 are 
represented by a single icon 163 because the object positions are 
the same whereas the icons 161 and 162 are presented in the same 
15 size since the length of shooting time of the images with the image 
numbers Al and A3 are the same. 

FIG. 12 shows a screen to be displayed by the display device 
110 based on the reproduced image information shown in FIG. 10. 

The second screen example 170 shown in FIG. 12 is an 
20 example of a display of the image selection screen to which changes 
are added by the image reproducing device 100 in order to display 
the icons larger according to a length of time taken for reproducing 
the image. 

In the second screen example 170 outputted based on the 
25 reproduced image information, the image with the image number Al 
is represented by an icon 171, and the images with the image 
numbers A2 and A4 are represented by an icon 172. The icon 171 
is presented larger than other icons because the length of 
reproduction time of the image with the image number Al is the 
30 longest, as shown in FIG. 10. 

Here, the image reproducing device 100 switches the display 
to the screen as shown in FIG. 13 in receiving a selection of an image 
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to be reproduced, when the icon 163 is selected. 

The third screen example 180 shown in FIG. 13 is an example 
of a display of the image selection screen when the images taken in 
the same position are presented on the screen. 
5 A title box 184 displayed on the third screen example 180 

presents the titles for the image numbers A2 and A4. Namely, 
when the icon 163 shown in FIG. 11 is selected, the title of the image 
indicated by the icon is presented in the title box 184. The title of 
the image is presented in the title box 184 according to a length of 

10 shooting time, an order of the names of the titles, a thumbnail 
presenting the image, or the like. Shooting information such as 
date and time of shooting or reproduction information like 
reproduction frequency may be used for rearrangement of the titles. 
The image reproducing device 100 can thus employ not only 

15 the display method of displaying the icons indicating a shooting 
position and an object position, and changing the size, form, color 
and the like of the icons according to the shooting information and 
the reproduction information, but also the display method of 
distorting a part of the map. FIG. 14 shows an example of a screen 

20 when the map data is changed according to degrees of moving trace. 

FIG. 14 is a diagram showing an example of the display 
method when the shooting location has moved in sequence. In the 
fourth screen example 190 shown in the upper left of the diagram 
presents routes 193, 194 and 195 as well as lines 191 andl92 

25 indicating a moving trace. The display on the fifth screen example 
190a shown in the right of the diagram can be obtained by changing 
the map data of the fourth screen example 190 according to the 
degree of moving route. In the fifth screen example 190a, the 
route (here, routes 194a and 195a) frequently used as a moving 

30 trace is emphasized with thick lines. Thus, the routes may be 
presented with an emphasis using wider lines or the line 192 
indicating a moving trace may be presented using a thicker line. 
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Thus, with the use of the image reproducing device 100 
according to the first embodiment, the user can grasp visually and 
intuitively a position in which the image is shot and a position of the 
object using the icons indicated on the map by reading a shooting 
5 position and an object position from a camera or the like and 
displaying it together with the map on the image selection screen. 

The image reproducing device 100 measures a length of 
shooting time, counts the number of times shooting in the same 
position as well as reproduction information of the image and stores 
10 them for display. When the icons on the map shown on the image 
selection screen are displayed, the user can also search out a 
desired image easily because a preference on the images can be 
presented visually by changing the color, size, form, or the like, of 
the icons according to the shooting frequency, the length of shooting 
15 time, the reproduction frequency and the length of reproduction 
time. 

(Second Embodiment) 

Next, the image recording/reproducing device according to 
20 the second embodiment of the present invention will be described. 

FIG. 15 is a block diagram showing the hardware construction 
of the image recording/reproducing device according to the second 
embodiment. 

As shown in FIG. 15, the image recording/reproducing device 
25 200 includes a positioning sensor 205, a direction sensor 206, a 
ranging sensor 207, a camera unit 208 and a display unit 210, in 
addition to the structure of the image reproducing device 100 
according to the first embodiment described above, and can be 
realized with a portable digital video camera or the like. 
30 The positioning sensor 205 is a sensor to detect a position of 

the camera in the image recording/reproducing device 200 and can 
be realized with a positioning device such as a GPS (Global 
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Positioning System). 

The direction sensor 206 is a sensor to detect a direction of 
the camera at the camera unit 208 in the image 
recording/reproducing device 200 and can be realized with a gyro 
5 sensor or an electronic compass placed in the image 
recording/reproducing device 200. 

The ranging sensor 207 is a sensor to measure a distance 
between the position of the image recording/reproducing device 200 
and the object and can be realized with an infrared sensor or a 
10 phase-contrast sensor. 

The camera unit 208 can be realized with a shooting device 
equipped with an image pickup device such as a Charge Coupled 
Device (CCD) or the like. 

The display unit 210 is a display device such as a finder or an 
15 LCD monitor of the camera. 

FIG. 16 is a block diagram showing a functional structure of 
the image recording/reproducing device 200 according to the 
second embodiment. 

As shown in FIG. 16, the image recording/reproducing device 
20 200 includes a recording unit 220 having a function as an image 
recording device like a digital still camera or the like and a 
reproduction unit 230 having the same function as the functional 
structure of the image reproducing device 100 according to the first 
embodiment described above. 
25 The recording unit 220 further includes a camera 

position/direction detection unit 221, an object distance detection 
unit 222, an object position calculation unit 223, an object search 
unit 224, a candidate object display selection unit 225, a camera 
position correction unit 226, a camera parameter setting unit 227, a 
30 camera operation control unit 228 and a shooting unit 229. 

The reproduction unit 230 includes a planned shooting 
location setting unit 237, a planned shooting location display unit 
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238 in addition to the functional components of the image 
reproducing device 100 according to the first embodiment described 
above. The reproduction unit 230 obtains the image data from the 
recording unit 220, therefore, the image input unit 131 in the image 
5 reproducing device 100 according to the first embodiment is 
excluded from the components. 

The camera position/direction detection unit 221, a 
processing unit for detecting a position and a direction of the image 
recording/reproducing device 200, can be realized with the 
10 positioning sensor 205 and the direction sensor 206 shown in FIG. 
15. 

The object distance detection unit 222, a processing unit for 
measuring a distance between the image recording/reproducing 
device 200 and the object, can be realized with the ranging sensor 

15 207 shown in FIG. 15. The object distance detection unit 222 
measures the distance by obtaining a range of a focal length either 
after detecting an aperture value at the camera and directly reading 
a depth of field corresponding to the lens or after reading the depth 
of field with reference to a table in which the depth of field is written. 

20 The object position calculation unit 223, a processing unit for 

figuring out an absolute position of the object, obtains the position 
of the object using the distance between the image 
recording/reproducing device 200 and the object measured by the 
object distance detection unit 222 as well as the position and the 

25 direction of the image recording/reproducing device 200 detected 
by the camera position/direction detection unit 221. 

FIG. 17 is a diagram describing processing for obtaining the 
object position. Here, the processing is presented using a 
two-dimensional model. 

30 When the camera position/direction detection unit 221 

detects that the position of the image recording/reproducing device 
200 is C 0 (x 0 , yo) and the direction of the camera is 9 0 and the object 
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distance detection unit 222 detects the focal length as Z, the object 
position calculation unit 223 obtains the position of the object as C 0 
(x P/ y p ) using the following equations: 
X p = Z x cos (G 0 ) + x 0 
5 y p = Z x sin (6 0 ) + yo 

Here, the example is shown using a two-dimensional model, but it 
can be expanded to a three-dimensional one. In that case, the 
same calculation can be made by presenting a direction of the 
camera in two angles and using a three-dimensional coordinate. 

10 The object search unit 224 is a processing unit for inquiring 

names or sizes of the building of the object located in the position 
calculated by the object position calculation unit 223. Here, the 
object search unit 224 searches for, via a network, the names and 
the sizes to an external server apparatus that stores the building 

15 database in which the building information is described. The 
building database may be stored inside the image 
recording/reproducing device 200. 

In the case of using the example shown in FIG. 17, the object 
search unit 224 searches the building database for the names and 

20 sizes of the building or the object corresponding to the object 
position P 0 (x p , y p ) calculated by the object position calculation unit 
223. 

Here, when a focusing range is presented as Z n ^Z^Z f 
reflecting a depth of field, the object search unit 224 searches for a 
25 building included in a rectangular presented as (Z f — Z n ) with P 0 in 
the center, as a candidate for an object. Namely, the object search 
unit 224 searches for the building or the object corresponding to the 
position P(x, y) which satisfies the following equations: 
Z n x cos (9 0 ) + x 0 ^ x ^ Z f x cos (8 0 ) + x 0 
30 Z n x sin (8 0 ) + y 0 ^ y ^ Z f x sin (8 0 ) + y 0 

The result of detecting plural candidate objects obtained by 
the object search unit 224 is shown in FIG. 18. FIG. 18 shows a 
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range focused by the camera using a hatched rectangle 250 and also 
shows that three candidate objects P p , Ql and Q2 are presented 
within the focusing range. 

The object search unit 224 searches the building database for 
5 respective candidate objects P p , Ql and Q2 and extracts the building 
information such as name, size, and others concerning the building. 
In this case, it is assumed that the depth of field is reflected on the 
values Z n and Z f which define the focusing range, however, these 
values may be received from the user via the I/O204 in the image 

10 recording/reproducing device 200. 

The candidate object display selection unit 225 is a 
processing unit for displaying a list of candidate objects searched 
out by the object search unit 224 on the display unit 210 such as a 
finder or an LCD monitor of a camera. Furthermore, the candidate 

15 object display selection unit 225 urges the user to select a desired 
object from among the candidate objects listed up for display. In 
the example shown in FIG. 18, P p , Ql and Q2 are searched out as 
candidate objects and the names of these buildings or the labels are 
displayed for the user to select a building to be an object of 

20 shooting. 

Here, the I/O 204 used by the user to select an object can be 
a directly — operable touch panel device placed on the monitor, a 
cursor, a switch, or the like. 

The camera position correction unit 226 is a processing unit 
25 for correcting a position of the image recording/reproducing device 
200, that is, a camera position, with the use of the position of the 
object calculated by the object position calculation unit 223 or the 
position of the object selected by the candidate object display 
selection unit 225 and feeding back the camera position to the 
30 camera position/direction detection unit 221 after the correction. 

FIG. 19 is a diagram for explaining processing of correcting a 
camera position. 
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The amount of correction for the camera position is presented 
as AC C ( Ax c , Ay c ) while the amount of correction for the direction of 
the camera is represented as A8 C . The position of the selected 
object is presented as Pi (xi, yi) whereas the camera position is 
5 presented as C = C C +AC C and the direction of the camera is 
presented as 0 = 8 0 +A8 c . A8 c can be obtained using the following 
Formula 1. 



x 0 — Z x cos8 and Ay c = y x — y 0 -Z x sine. Consequently, the camera 
position can be corrected to C 0 (x' 0/ y'o) as shown in FIG. 19. 

The camera parameter setting unit 227 is a processing unit 
for calculating the camera parameters for shooting the object using 

15 the size of the object extracted by the object search unit 224 or the 
size of the object selected by the candidate object display selection 
unit 225. For explaining the camera parameters, a picture angle is 
taken here as an example. FIG. 20 is a pattern diagram showing a 
camera position, an object position and a size of the object. In FIG. 

20 20, width and depth of an object 260 is presented respectively by W 
and D, whereas Z presents a distance between the camera and the 
object, and a presents an angle formed by the camera and the 
object. Here, the picture angle in a horizontal direction can be 
obtained using the formula 2 below based on the camera position, 

25 the object position and the size of the object. 



[Formula 1] 



A0 C = tan 




V x i " x o J 



10 



AC C can be gained using the following equations Ax c = Xi — 
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[Formula 2] 



tan (j- ) 

Thus, the picture angle in a horizontal direction is set to an 
appropriate value in consideration of the camera position, the object 
5 position and the size of the object. 

The camera operation control unit 228 controls the operation 
of the camera according to the camera parameters set by the 
camera parameter setting unit 227. FIG. 21 shows an example of 
displaying the object 260 to be shot after the angle correction 

10 described in FIG. 20. 

The shooting unit 229, a processing unit for shooting an 
object according to the operation performed by the camera 
operation control unit 228, can be realized with the camera unit 208 
shown in FIG. 15. The shooting unit 229 stores the shot image as 

15 image data in the image/shooting information storage unit 231. In 
this case, the shooting unit 229 stores the camera parameters set at 
the time of shooting and the length of shooting time or the like as 
shooting information together with the image data in the 
image/shooting information storage unit 231. 

20 The image/shooting information storage unit 231 is a storage 

device such as a memory or a hard disk for storing the position and 
direction of the camera either detected by the camera position 
detection unit 221 or corrected by the camera position correction 
unit 226 and then fed back to the camera position/direction 

25 detection unit 221, the distance to the object detected by the object 
distance detection unit 222, the position of the object calculated by 
the object position calculation unit 223 and the image data shot by 
the shooting unit 229. 

The display method selection unit 233 receives the settings 

30 such as whether or not to display planned shooting locations and 
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whether or not to distinguish the attributes like size, color and 
others of the icon for the display of the planned shooting locations 
where the shooting is not performed yet from the icon indicating the 
location where shooting is realized. 
5 The planned shooting location setting 237 is a processing unit 

for receiving an input for the planned shooting locations from the 
user via the I/O 204 in the image recording/reproducing device 200 
and setting the planned shooting locations. The I/O 204 used here 
can be a touch panel and buttons set on the camera or an input 

10 interface to be used exclusively for inputting positions. The 
position information to be inputted can be the names of the planned 
shooting locations and coordinates for indicating the positions using 
latitude and longitude, or the like. 

The planned shooting location display unit 238 is a processing 

15 unit for displaying the icon indicating the planned shooting location 
set by the planned shooting location setting unit 237 by 
superimposing it on the map outputted from the display unit 210 in 
the image recording/reproducing device 200. When it is set to 
change the display of the icon indicating the location where shooting 

20 is not performed in the display method selection unit 233, the icon 
is changed and then displayed according to the icon change 
attributes. 

Other functional components included in the reproduction 
unit 230 are almost the same as the components shown in the first 

25 embodiment described above, therefore, the description is 
abbreviated here. 

Next, the processing operation performed by the image 
recording/reproducing device 200 thus constructed will be explained 
with reference to FIG. 22. The following describes only correction 

30 processing of the camera position and the camera parameters which 
are unique to the image recording/reproducing device 200 according 
to the second embodiment. 
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FIG. 22 is a flowchart showing a flow of the correction 
processing performed by the image recording/reproducing device 
200 according to the second embodiment. 

Firstly, the camera position/direction detection unit 221 
5 detects a camera position and a camera direction (S200). 

The object distance detection unit 222 detects a distance 
between the image recording/reproducing device 200 and an object 
(S202) and calculates an object position using the detected distance 
(S204). 

10 The object search unit 224 subsequently refers to the 

calculated object position and searches for a building or an object to 
be a candidate for shooting from the building database (S206). 
Here, the object search unit 224 searches for a building or an object 
based on whether or not a candidate object is found within a 

15 predetermined spatial range having the object position in the center 
(the rectangular shown in FIG. 18). 

When plural candidate objects are found within the range 
mentioned above (Y in S208), the object search unit 224 extracts 
the names and the sizes and others of the candidate objects whereas 

20 the candidate object display selection unit 225 lists them up for 
display (S210) and receives a selection of a desired object from the 
user (S212). 

When plural candidate objects are not found within the range 

(N in S208), or after receiving the selection of the desired object 
25 from the user, the camera position correction unit 226 corrects the 

camera position based on the position of the object either included 

in the range or selected (S214). 

Then, the camera parameter setting unit 227 sets the camera 

parameters at the time of shooting based on the size and others of 
30 the object (S216) and the camera . operation control unit 228 

operates the camera according to the camera parameters that are 

already set (S218). 
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After that, the shooting unit 229 shoots the object and the 
image recording/reproducing device 200 terminates the processing 
(S220). 

The following describes in detail a screen to be outputted for 
5 display by the display unit 210 in the image recording/reproducing 
device 200. The following describes only the output display screen 
that is unique to the image recording/reproducing device 200 
according to the second embodiment. 

FIG. 23 is a diagram showing an example of candidate object 
10 selection screen. 

An example of the candidate object selection screen 270 
shown in FIG. 23 is an example of the screen created by the 
candidate object display selection unit 225. In the diagram, 
objects 271~273 are shown as candidate objects. Here, the image 
15 recording/reproducing device 200 receives an operation of selecting 
the object 273 from the user via the I/O 204 such as a touch panel 
and buttons of a monitor and then switches the display to the screen 
shown in FIG. 24. 

FIG. 24 is a diagram showing an example of the screen after 
20 deciding an object. 

The example of the screen after object determination 280 is 
an example of the screen to be displayed when the object is shot and 
information such as the camera parameters which are set as well as 
the names of the objects is displayed together with the object. 
25 FIG. 25 is a diagram showing an example of the display of the 

planned shooting location. 

The example of the planned shooting location display screen 
290 shown in FIG. 25 is an example of the screen created by the 
planned shooting location display unit 238. In the diagram, three 
30 planned shooting locations are displayed respectively using the 
icons 291~293. Only the icon 293 indicating a planned shooting 
location 3 is displayed larger than the icons 291 and 293 because 
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shooting is not performed yet in a planned shooting location 3 
indicated by the icon 293. Also, the symbol partly covered by the 
icon 292 and the characters "planned shooting location 2" is a map 
symbol indicating a shrine whereas the symbol presented besides 

5 the planned shooting location 3 or partly covered by the icons 292 
and 293 is a map symbol indicating a temple. The camera 
position/direction detection unit 221 detects each position of the 
image recording/reproducing device 200 for each predetermined 
period of time and continues to memorize it, regardless of whether 

io the image is shot or not. 

Thus, the image recording/reproducing device 200 according 
to the second embodiment, which is a combination of the image 
reproducing device 100 described in the first embodiment and an 
image recording device such as a camera and the like, detects a 

15 camera position, a camera direction and camera parameters and 
calculates an object position based on them. The image 
recording/reproducing device 200 then detects whether or not plural 
candidate objects are found either within a range of the depth of 
field having the calculated object position in the center or within a 

20 predetermined spatial range. When there is a possibility to find 
plural candidate objects, the image recording/reproducing 
apparatus 200 can surely determine the object desired by the user 
and hold a correct position of the object by receiving a selection of 
an object out of the candidate objects from the user. 

25 The camera position, the camera direction and the camera 

parameters are corrected based on the position of the selected 
object. Also, the picture angle appropriate for shooting is 
calculated using the size of the object and the camera parameters. 
Therefore, a framing in shooting an image can be facilitated. 

30 The planned shooting locations can be indicated on the map 

for the user since the image recording/reproducing device 200 
receives an input of the planned shooting location and displays the 
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shooting sites on the map. The user cannot miss the shooting 
because the sites where the shooting is not performed are displayed 
visually with the icons. The user can further record the moving 
trace because the camera position is recorded continuously even 

5 when shooting is not operated. 

As is the case of the image reproducing device 100 according 
to the first embodiment, user's preference can be displayed visually 
by displaying a favorite image larger or in other manners because 
the position of shooting an image and the object position included in 

10 the image is displayed using the icons indicating the image data on 
the map. 

Thus, the image reproducing device and the image 
recording/reproducing device according to the present invention are 
described based on each of the embodiments^ however, the present 

15 invention is not limited to these embodiments. 

For example, it is explained in each of the above-mentioned 
embodiments that the image reproducing device and the image 
recording/reproducing device place the icons indicating the image 
data on the image selection screen having a map in the background. 

20 However, they may be placed on the image selection screen having 
an image in the background, instead. In this case, the image 
reproducing device and the image recording/reproducing device 
may keep, in a memory or the like, a table correlating virtually a 
pixel position of the image with a position in the map and place the 

25 icons on the image selection screen according to the table. 

For example, the image reproducing apparatus 100 may 
incorporate a display device 110 shown in the first embodiment. 

In the second embodiment, the object search unit 224, the 
candidate object display selection unit 225, the camera position 

30 correction unit 226, the camera parameter setting unit 227 and the 
camera operation control unit 228 included in the recording unit 220 
or the planned shooting location setting unit 237 and the planned 
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shooting location display unit 238 included in the reproduction unit 
230 are only the options and not indispensable for the present 
invention. 

In addition, the case of combining a image recording device 
5 and an image reproducing device is described in the second 
embodiment, however, a single image reproducing device may not 
only display an image selection screen using information on the 
image such as shooting position, reproduction frequency, and others, 
but also be connected to another apparatus so as to display an 
10 image selection screen using such information. 

Nevertheless, the screen examples shown in the above 
embodiments are just for examples, and the display shall not be 
limited to them. 
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